Bayesian Nonparametric Models

‘ We have been looking at models that posit latent structure in high dimensional data. We use the posterior to uncover that structure. ‘ The two main types are mixtures (and variants, like mixed-membership) and factor models (like PCA, factor analysis, and others). ‘ A nagging concern for these methods is model selection—how do I choose the number of mixture components? the number of factors? ‘ This is a field unto itself. You have probably heard of cure-all techniques, like AIC, BIC, and DIC. It can be fitted, for example, with cross-validation or Bayes factors. Sometimes existing theory or problem constraints inform the number of components. ‘ Bayesian nonparametric modeling (BNP) provides an alternative solution. BNP methods have enjoyed a recent renaissance in machine learning. ‘ Loosely, BNP models posit an “infinite space” of latent structure where (in the generative process) the data only uses a finite part of it.